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APPARATUS AND METHOD FOR DETECTING A MOVING 
OBJECT IN A SEQUENCE OF COLOR FRAME IMAGES 

5 

The present invention relates to an apparatus and method for detecting 
a moving object in a motion picture, and more specifically, to an apparatus and 
method for detecting a face region from a motion picture by using the color and 
motion information of the face region. 

10 

Background Art: 

Detecting a face region from a motion picture is one of the prerequisite 
steps in face recognition. Conventional schemes for detecting a face region 
15 have not been used widely, since those schemes are affected by background 
images, and the size and orientation of the face. 

Detection of face regions may be performed by using the information of 
the shape, color, or motion of the faces. 

Using shape information for detecting a face region may be performed 
20 by measuring variation of gray levels and applying the measured values to a 
priori information of the face. However, the scheme should be applied only 
to images containing front side of a face, and the detection result is largely 
affected by background images, and the size and orientation of the face. 

The scheme of using color information suffers from racial deviations of 
25 face colors, because the scheme detects a face region by using the inherent 
color of human faces. Further, the scheme requires large amount of data 
processing, since it uses much more information than the scheme of using shape 
information. However, it becomes more applicable than the scheme of using 
shape information, as hardware technology has recently developed. 
30 The motion information may also be used for detecting face regions in 



wo 00/08586 

- 2 - 

motion pictures wherein an object, i.e. a face, is moving. 
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Disclosure of Invention: 

5 It is therefore a principal object of the invention to provide an apparatus 

and method for detecting a moving object wherein the detection is not affected 
by backgroimd images, and the size and orientation of the moving object. 

It is another object of the invention to provide an apparatus and method 
for detecting a moving object including a face region wherein the detection may 

10 be performed faster than the prior art. 

In accordance with one aspect of the present invention to achieve the 
aforementioned object, an apparatus for detecting a moving object in a sequence 
of color frame images is provided. The color frame images have a plurality of 
pixels, each having three color components. The apparatus comprises a color 

15 normalizer for normalizing color components of the color frame image to 
produce a normalized frame image, a color transformer coupled to the color 
normalizer for color transforming the normalized frame image to a first color 
transformed frame image, the first color transformed frame image having 
intensity levels such that pixels corresponding to said moving object are 

20 emphasized, a frame delay coupled to the color transformer for delaying the 
first color transformed frame image by one frame to produce a second color 
transformed frame image, and a motion detector coupled to the color 
transformer and the frame delay for detecting the motion of the moving object 
and further intensifying the intensity levels of the first color transformed frame 

25 image based on the detected motion. 

Brief Description of Drawings: 

The present invention will become more apparent upon a detailed 
30 description of the best mode for carrying out the invention as rendered below. 
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In the description to follow, references will be made to the accompanying 
drawings, where like reference nimierals are used to identify like parts in the 
various views and in which: 

Fig. 1 shows a block diagram of the overall system for detecting a 
5 moving object in a color motion picture in accordance with the present 
invention; 

Fig. 2 shows a block diagram of the color normalizer 100 of Fig. 1; 
Fig. 3a shows a histogram of normalized color components in a typical 
face image. 

10 Fig. 3b shows a histogram of normalized color components in a face 

images modeled by using 2-dimensional Gaussian distribution function. 

Fig. 4 shows a block diagram of the color transformer 200 of Fig. 1; 
Fig. 5 shows results of color transformation by using GFCD modeling 
of a typical face image; 

15 Fig. 6 shows a block diagram of the motion detection block 400 of Fig. 

1. 

Fig. 7 shows motion detection results of two successive images which 
have a rectangular object. 

Fig. 8 shows motion detection results of two successive images having 
20 a human face. 

Best Mode for Carrying out the Invention: 

Fig. 1 shows a block diagram for illustrating the overall system for 
25 detecting a moving object in a color motion picture in accordance with the 
present invention. The system comprises a color normalizer 100 for 
normalizing color components of a current frame image of the color motion 
picture; a color transformer 200 for color transforming the normalized frame 
image to a color transformed frame image; a frame delay 300 for temporarily 
30 storing the color transformed frame image from the color transformer 200 to 
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delay the color transformed frame image by one frame; and a motion detector 
400 for receiving the color transformed current and previous frame images 
from the color transformer 200 and the frame delay 300 and detecting the 
motion of the moving object. The color transformed frame image, which is 

5 produced by the color transformer, has intensity levels such that pixels 
corresponding to the moving object are emphasized. The motion detector 400 
uses the detected motion of the moving objects to further intensify the intensity 
levels of pixels of the moving object. 

Referring now to Fig. 2, the operation of the color normalizer 100 of 

10 Fig. 1 will be explained. 

In color model of RGB color space, for example, any color information 
Q may be represented by three color components, i.e., red, green, and blue 
color components R, G, and B, having different wavelengths. In other words, 
the color information Q may be given by Q = (R, G, B). A luminance value 

15 Y may be defined by sum of the three color components. The luminance value 

Y is a measvure of a visible radiant energy which produces the sensation of 
brightness. In order to prevent brightness of an image from affecting the color 
components of an image, the color components of each pixel must be 
normalized by luminance value Y of the pixel. 

20 Fig. 2 shows a block diagram of the color normalizer 100. The color 

normalizer 100 comprises a luminance extractor 130, and two normalizers 110 
and 120. The luminance extractor 130 evaluates a luminance value Y by 
summing red, green, and blue color components R, G, and B for each pixel of 
a frame image, i.e., Y = R + G + B, and outputs it. Then, the normalizer 

25 110, for example, red component normalizer receives the Y values and outputs 
normalized color component r by using the luminance value Y and red color 
component R, wherein r is given by (R/Y) * 255. Similarly, the normalizer 
120, for example, green component normalizer receives the luminance value 

Y and outputs normalized color component g which is given by g = (R/Y) * 
30 255. Hereinabove, we assumed that the luminance value has depth of 3 bits. 
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and the most bright pixel has the luminance value of 255. By the 
normalization, any color information Q may be represented by two normalized 
color components r and g, i.e. Q = (r, g), since the three normalized color 
components r, g, and b has the relationship of r + g + b = 255. In this way, 
5 the color model of the normalized color space can represents 256^2 colors, 
while the color model of RGB color space represents 256^ colors. 

Fig. 3a shows histogram of normalized color components in a typical 
face image. In Fig. 3a, two horizontal axis represents two normalized color 
component values r and g, and the vertical axis represents number of pixels 

10 having normalized color components of r and g in a typical face image. 
Distributions of the normalized color components are similar for all face images 
regardless of backgrounds and brightness. 

Fig. 3b shows histogram of normalized color components in face image 
which is modeled by 2-dimensional Gaussian distribution function. In this 

1 5 embodiment of the invention, the modeling is performed by using 2-dimensional 
Gaussian distribution as shown in Fig. 3b. The Gaussian model for color 
distribution in face images is called Generalized Face Color Distribution 
(GFCD). GFCD is defined by mean m and variance E^, in other words, GFCD 
is represented by GF(m, E^) t^js representation, m = (/r, /g) is the center 

20 point of the 2-dimensional Gaussian distribution, wherein /r and /g are mean 
values of normalized color components for all pixels in a face image. TP- 
represents a covariance matrix of normalized color components in a face image. 
In a preferred embodiment, a GFCD for face images may have m = (105, 95) 
and of a, = 20 and = 15. By using the GFCD model, an input color 

25 iinage containing a face may be transformed to a gray-level image where pixels 
having color of the face are emphasized. 

The color transformation using a color distribution model transforms 
normalized color components of pixels to values which represent the proximity 
of the color components to the mean m = (/r, /g) of the Gaussian distribution 

30 in the normalized color space. The color transformation is defined as follows: 
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f : i?^— i?^ 

Z(^, y) =GF(r (X, y) , gr(x, y) ) (x, y) ei Eq.\ 

.wherein (x, y) is a coordinate of a pixel in the face image, g(x, y) and r(x, y) 
are normalized color components of the pixel at the coordinate (x, y), and GFQ 
is Generalized Face Color Distribution function. The color transformation 
using GFCD produces values which are proportional to the proximity of the 
5 color of the pixel to a hiunan face color. 

Referring to Figs. 4 and 5, the operation of the color transformers 200 
of Fig. 1 will be explained. 

Fig. 4 shows a block diagram of die color transformer 200. Color 
transformer 200 comprises blocks 210 and 220, two normalization block 230 

10 and 240, and output block 250. Block 210 receives the normalized color 
component r from the normalizer 110 and compares it with which is 
predetermined in accordance witii a particular GFCD model. If the normalized 
color component r is within three times of a, around /r, then, the block 210 
ouQjuts r' = 1 r - /r I . Otherwise, the block 210 outputs a very large value 

15 in order to make Zr = 0. Blocks 230 receives the output value r' and outputs 
transformed value Zr by referring to a look-up table LUT which contains 1- 
dimensional normalized Gaussian number GF(0, a^). Similarly, block 220 
receives the normalized color component g and compares it with o^. If the 
normalized color component g is within three times of around /g, then, the 

20 block 220 outputs g' = | g - /g | . Otherwise, the block 220 outputs a very 
large value in order to make Zg = 0. The block 240 receives die output value 
g' and outputs transformed value Zg by referring to a look-up table LUT which 
contains 1 -dimensional normalized Gaussian number GF(0, a^'). Output block 
250 receives Zr and Zg values and evaluates Z = (Zr * Zg) * 255 and outputs 

25 Z. Fig. 5 shows results of color transformation by using GFCD modeling of 
a typical face image. As can be seen in the result image of Fig. 5, areas 
having human face color are emphasized. 

If any background images have color which is similar to the color of 
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hirnian face, the above method may emphasize those backgroimd images as well 
as face area. In order to overcome this problem, the present method uses 
motion information of hmnan face. In general, only the human face area is 
moving while background images remain still. Thus, the human face area may 
5 be outstandingly distinguished from background images, if motion information 
of the face area is utilized. 

Two methods for extracting motion information from two successive 
frame images are well known in the art. They are region based method and 
feature point based method. Most applications in the art use the region based 
10 method rather than the feature point based method, since the latter generally 
requires post-processing which uses interpolation technique. 

In a typical region based method, the inter-frame differences of intensity 
levels of pixels adjacent a particular pixel are summed in order to measure the 
motion of the particular pixel. This method is known as Accumulated 
15 Difference Measure (ADM) method. Since the ADM method uses the 
summation of intensity differences of neighbor pixels, it detects a motion 
robustly against small noisy change of neighbor pixels. However, pixels having 
small differences are ignored by averaging effect, and the measurements are 
largely affected by the determination of threshold value. 
20 Thus, the present invention preferably performs motion detection by 

counting pixels whose inter-frame differences are larger than a threshold value 
in a window having predetermined size. This motion detection measure will 
be called Unmatching Pixel Count (UPC) measure and may be written as 
follows: 

UPCix. y, t) = UU.j.t) Eg. 2 

i=x-N j=y-N 

25 . ^ [1, if\Z(i.j. t)-Z(i.j.t-l)\>Th 

U(i. J. t) = 

{0, otherwise 

wherein Th is a threshold value for determining whether the intensity values of 
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pixels in two successive frame images are matched. In the UPC method, since 
matching is tested in a pixel by pixel manner and the results are accumulated, 
the averaging effect is reduced such that overlapping area causing a small 
intensity change is clearly captured. 

5 In order to detect motion of a color image, the color information 

explained hereinabove may be used. It may be recognized by those skilled in 
the art that the intensity level of each pixel in the color transformed image, 
which was transformed by using the color transformation method of GFCD 
modelling, represents a probability diat die pixel is in a face area. The color 

10 information of the color transformed image may be incorporated in Eq. 2 by 
weighting the counts with the intensity level of the pixel of the color 
transformed image. This new motion detection measure will be referred to as 
Weighted Unmatching Pixel Count (WUPC). The WUPC(x, y, t) is evaluated 
by performing fuzzy- AND operation between the UPC measure UPC(x, y, t) 

15 and the intensity level of the color transformed image Z(x, y, t). The WUPC 
measure is given by following equation: 

i=x+N j~y+N 

WUPCU.y. t) =Z{K,y, t) ^ 52 E UU.j.t) Eg. 3 



a. J. = I J 



u(i j t) = r' 'f^^(''j' t)-z(ijj-i) \ >Th 

otherwise 



20 wherem the operator ® is fuzzy- AND operator. This WUPC measure 
emphasizes motion of the pixels having color which is sunilar to color of a 
human face, while deemphasizing motion of the other pixels, i.e., pixels of 
background images. 

In Eq. 3, the threshold value Th may be obtained adaptively by using the 

25 transformed image value Z(x, y, t) for each pixel. In other words, the Th 
becomes small for pixels which are in a face area and large for other pixels. 
In this way, the WUPC measure is more sensitive to motion of pixels in a face 
area than motion of the other pixels, such that even small motion of face area 
may be detected. In a preferred embodiment of the present invention, the 

30 threshold Th may be obtained by using Sigmoid function as follows: 

Slope of Sigmoid function becomes steep as Q is lowered. Thus, Sigmoid 
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ThiZ) = 



255 



Eq. 4 



1+e 



Z(x, y, t) -255/2 
O 



function is a lot like a step-ftinction shape for small Q. It is appropriate to use 
Sigmoid function as a function for determining threshold Th of Eq. 3, since the 
Sigmoid function outputs small threshold Th for a large mput value and large 
Th for small input value. Further, the Sigmoid function is nonliuear in that the 

5 function does not respond abruptly to maximum or minimum input values. 
This method for adaptively counting unmatched pixels will be called Adaptive 
Weighted Unmatching Pixel Count (AWUPC) measure. 

Fig. 6 shows a block diagram of the motion detection block 400 of Fig. 
1. The motion detection block 400 comprises line buffers 410 and 420 for 

10 temporarily storing color transformed images and outputting intensity levels of 
pixels in a window of a predetermined size; an adaptive threshold value 
generator 440 for generating threshold th in response to intensity level of a 
central pixel in the window; an unmatching pixel value generator 430 for 
receiving the threshold Th and present and previous frames of the color 

15 transformed image from line buffers 410 and 420 to determine whether the 
pixels in the window of the present frame match with the pixels in the window 
of the previous frame by evaluating unmatching pixel value U(x, y, t) of Eq. 
3; an unmatching pixel counter 450 for counting the pixels which are 
determined to be unmatching pixels in the unmatching pixel generator 430; and 

20 a fuzzy-AND operator 460 for performing fixzzy-AND operation on the outputs 
of the unmatching pixel counter 450 with the intensity level Z(x, y, t) of the 
pixel of the color transformed image from the line buffer 420. The line buffer 
420 outputs intensity level Z(i, j, t) in a window of the present frame, while the 
line buffer 410 outputs intensity level Z(i, j, t-1) in a window of the previous 

25 frame of the color transformed image. The window size is (2N -f 1) x (2N -h 
1) such that the line buffers 410 and 420 output intensity levels of (2N + 1) x 
(2N -f 1) pixels. The threshold value generator 440 receives the intensity level 
of the central pixel Z(x, y, t) among the window of the present frame and 
evaluates a threshold value Th according to Eq. 4. The unmatching pixel 

30 generator 430 receives outputs from die line buffers 410 and threshold value 
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Th from the threshold value generator 440. Then, the unmatching pixel 
generator 430 evaluates the u(i, j, t) of Eq. 3 for a central pixel Z(x, y, t) of 
the window. The unmatching pixel counter 450 receives the outputs of the 
unmatching pixel generator 430 and counts the number of unmatching pixels. 
5 The fuzzy-AND operator 460 outputs a color motion information by performing 
a fuzzy-AND operation between the intensity level of the central pixel Z(x, y, 
t) from the line buffer 420 and the outputs from the unmatching pixel counter 
450. 

Referring to Figs. 7 and 8, motion detection results of the present 

10 invention will be compared with those of the prior art. Fig. 7 shows motion 
detection results of two successive images which have a rectangular object. 
Fig. 7a shows the first images having a rectangular object with background 
images. Pixels in the rectangular object of Fig. 7a have random gray levels of 
200 - 230, while the gray levels of pixels in the background images are random 

15 values of 0 - 30. Fig. 7b shows the second image wherein the rectangular 
object is moved by 50 pixels in x and y directions. Fig. 7c shows the motion 
detection results according to the conventional ADM method. The areas, 
whose inter-frame differences of intensity levels are large, are emphasized. 
However, those areas where the rectangular objects of the two subsequent 

20 images are overlapped are not emphasized. Thus, the motion is not well 
detected by the conventional ADM method when the two objects of sebsequent 
femes are overlapped. Fig. 7d shows a motion detection result according to the 
UPC measure. In Fig. 7d, those areas where the rectangular objects of the two 
subsequent images are overlapped are also emphasized. The UPC measure can 

25 detect motion of a moving object better than the ADM method, since it is able 
to detect overlapping areas of a moving object. Fig. 7e shows motion detection 
result according to the WUPC measure. The areas of the moving object having 
a desired color in the first image are emphasized. Fig. 7f shows the motion 
detection results by using A WUPC measure according to the present invention. 

30 Fig. 8 shows motion detection results of two successive images having a human 
face. Figs. 8a shows GFCD color transformed image of the first images 
having a human face with background images. Fig. 8b shows GFCD color 
transformed image of the second image wherein the human face is moved 
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slightly. Fig. 8c shows the motion detection results according to the 
conventional ADM method. The motion is not well detected by die 
conventional ADM method as shown in Fig. 8c. Fig. 8d shows a motion 
detection result according to the UPC measure. Figs. 8e and 8f respectively 
5 show motion detection result according to the WUPC measure and AWUPC 
measure in accordance with the present invention. 
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Claims: 



1 . An apparatus for detecting a moving object in a sequence of color frame 

images, said color frame image having a plurality of pixels, each pixel having 
5 three color components, comprising: 

a color normalizer for normalizing color components of the color frame 

image to produce a normalized color frame image; 

a color transformer coupled to the color normalizer for color 

transforming the normalized color frame image to a first color transformed 
10 frame image, said first color transformed frame image having intensity levels 

such that pixels corresponding to said moving object are emphasized; 

a frame delay coupled to the color transformer for delaying the first 

color transformed frame image by one frame, said delayed first color 

transformed frame image being a second color transformed frame image; and 
15 a motion detector coupled to the color transformer and the frame delay 

for detecting the motion of the moving object and further intensifying the 

intensity levels of said first color transformed frame image based on the 

detected motion. 

20 2. The apparatus of Claim 1, wherein each pixel of the first color 
transformed frame image has an intensity level which is proportional to the 
proximity of the normalized color components of the pixel to the normalized 
color components of said moving object. 

25 3. The apparatus of Claim 2, wherein the intensity level of said each pixel 
of the first color transformed frame image and the normalized color 
components of the pixel have a relationship as follows: 



wherein (x, y) is a coordinate of said pixel in the normalized frame image, r(x, 
y) and g(x, y) are normalized color components of the pixel at the coordinate 
30 (x, y), and GF( ) is a 2-dimensional Gaussian distribution function. 



Z(x, y) =GF{r{x, y) , g{x, y) ) 



y) el 
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4. The apparatus of Claim 1 , wherein said motion detector comprises means 
for detecting the motion of each pixel by counting pixels adjacent said each 
pixel whose intensity level differences between said first and second color 
transformed frame images are larger than a threshold value, and 
5 wherein said intensity level of each pixel are further intensified by 

weightmg said intensity level in accordance with said detected motion of said 
each pixel. 



5. The apparatus of Claim 4, wherein said weighting is performed by 
10 fiizzy-AND operating said intensity level with said detected motion for said 
each pixel. 



6. The apparatus of Claim 4, wherein said threshold value is obtained by 
using a Sigmoid function as follows: 

ThiZ) = 

Zix, y, t) -255/2 

1 + e ^ 

15 wherein Z(x, y, t) is the intensity level of a pixel and Q is a predetermined 
parameter. 



7. A method of detecting a moving object in a sequence of color fi:ame 
images, said color frame image having a plurality of pixels, each pixel having 
20 three color components, comprising steps of: 

normalizing color components of the color frame image to produce a 
normalized frame image; 

color transforming the normalized frame image to a first color 
transformed frame image, said first color transformed frame image having 
25 intensity levels such that pixels corresponding to said moving object are 
emphasized; 

delaying the first color transformed frame image by one frame, said 
delayed first color transformed frame image being a second color transformed 
frame image; and 

30 detecting the motion of the moving object and further intensifying the 
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intensity levels of said fost color transformed frame image based on the 
detected motion, 

8 . The method of Claim 7 , wherein each pixel of the first color transformed 
5 frame image has an intensity level which is proportional to the proximity of the 

normalized color components of the pixel to the normalized color components 
of said moving object. 

9. The method of Claim 8, wherein the intensity level of said each pixel of 
10 the first color transformed frame image and the normalized color components 

of the pixel have a relationship as follows: 

Z{k, y) =GF{r{x, y) , gr(^, y) ) (x, y) eJ 

wherein (x, y) is a coordinate of said pixel m the normalized frame image, r(x, 
y) and g(x, y) are normalized color components of the pixel at the coordiaate 
(x, y), and GF( ) is a 2-dimensional Gaussian distribution function. 

15 

10. The method of Claun 7, wherein said motion detecting step comprises 
a step of detecting the motion of each pixel by coimtmg pixels adjacent said 
each pixel whose intensity level differences between said first and second color 
transformed frame images are larger than a threshold value, and 

20 wherein said intensity level of each pixel is further intensified by 

weighting said intensity level m accordance with said detected motion of said 
each pixel. 

1 1 . The method of Claim 10, wherein said weighting is performed by fuzzy- 
25 AND operatmg said intensity level with said detected motion of said each pixel. 

12. The method of Claim 10, wherein said threshold value is obtained by 
using a Sigmoid function as follows: 
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wherein Z(x, y, t) is the intensity level of a pixel and Q is a predetermined 
parameter. 
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FIG. 3a 
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FIG. 5a 



FIG. 5b 
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